A Three Parameter Underwater Image Formation Model

We developed an underwater image formation model that describes how light is absorbed and scattered by seawater and its constituents. We use the model to predict digital camera images of a reference target with known spectral reflectance at different distances and depths. We describe an inverse estimation method to derive three model parameters: phytoplankton absorption spectrum, chlorophyll concentration and the amount of colored dissolved organic matter or CDOM. The estimated parameters predict the spectral attenuation of light which can be used to color balance the images. In addition, parameter estimates can be used to monitor environmental changes turning a consumer digital camera into a scientific measurement device. Introduction The digital camera has become an accessory that most people take with them everywhere, including underwater. Sadly, they are often disappointed with the quality of their underwater images. Backscattered light reduces image contrast and wavelength dependent light absorption by water introduces color changes [1, 2]. No doubt the quality of underwater photography will improve as the low-light sensitivity of imaging sensors increases and as new image processing methods are introduced. Several underwater image correction algorithms operating on RGB images have been proposed [3, 4], but only a few methods analyze the data in the spectral wavelength domain [5, 6, 7]. In most cases, the goal of these algorithms is to improve color rendering, rather than infer biologically relevant quantities [7, 8, 9]. In this paper we consider how to derive scientific data from underwater camera sensor images in order to characterize the ocean seawater environment. We also illustrate how this data can be used to process and improve the perceived quality of underwater images. We developed an underwater image formation model to describe how light is absorbed and scattered by water and its constituents and how light is captured by the imaging sensor in a digital camera. We use our model to simulate the appearance of images captured by digital cameras and to relate the appearance to physically interpretable quantities, such as the type and amount of phytoplankton and other organic and inorganic matter in sea water [10]. We also use the insights we gained from these simulations to improve the way we process underwater images in order to produce more aesthetically pleasing photographs [7]. Our underwater image formation model is composed of three components. First, we use the underwater image formation model of Jaffe and McGlamery [11, 12] to describe light absorption and scattering in units of medium beam absorption and scattering coefficients. Second, we incorporate the results of oceanographic and biological research describing attenuation and scattering coefficients as functions of concentrations of fundamental constituents of sea water: phytoplankton, colored dissolved organic matter (CDOM) and non-algal particles (NAP) [10]. Third, we use a full camera simulation package (ISET, [13]) to produce simulated images of underwater targets. We use the underwater image formation model to predict the sensor data captured by a digital camera at a fixed distance and depth from a reference target with known spectral reflectance. With the appropriate parameter settings, we can reproduce the appearance of sensor images captured by real cameras in similar underwater environments. We wish to use the digital camera as a scientific instrument that can measure environmental factors, such as the type and concentration of phytoplankton and other material in the seawater. To accomplish this, we introduce an inverse estimation method that uses the camera sensor data to derive parameters that describe 1) the spectral absorption of light by phytoplankton, 2) the concentration of chlorophyll in phytoplankton, and 3) the amount of colored dissolved organic matter or CDOM. We use the inverse estimation method as a metric to evaluate how well any digital camera can be used to measure environmental parameters and consider how these measurements can also be used to improve the perceived quality of underwater images. Image formation model The measurement m produced by an imaging device is linearly related to device’s spectral sensitivity functions p(λ ) and the light radiance ρ(λ ) reaching the photodetector [14] m = ∫ p(λ )ρ(λ )dλ . (1) A ray of light traveling between the source and the scene interacts with the medium in two ways. First, some of the light may be absorbed by the medium, and thus the overall intensity of light is reduced. Second, the direction of propagation of a portion of the light ray may be changed in a phenomenon called scattering. As a consequence of these interactions the total radiance along a particular ray of light ρ(λ ) reaching an imaging device can be decomposed into two additive components; direct ρd(λ ) and backscattered ρb(λ ) [1, 11, 12] ρ(λ ) = ρd(λ )+ρb(λ ). (2) The direct component contains all the light rays that, having been emitted by a source, interact with a scene. The backscattered component represents all the light rays whose direction of propagation was changed by the medium before they reached the target, which means they are captured by the imaging device without interacting with the scene (Fig. 1). The McGlamery-Jaffe underwater image formation model [11, 12] describes how the absorption and scattering affect the ©2016 Society for Imaging Science and Technology DOI: 10.2352/ISSN.2470-1173.2016.18.DPMI-252 IS&T International Symposium on Electronic Imaging 2016 Digital Photography and Mobile Imaging XII DPMI-252.1 Figure 1: Direct and backscattered components in underwater imaging. Figure 2: The intensity of light traveling through a medium is reduced due to absorption and scattering. direct and backscattered radiance components. However, for uniform surfaces at a fixed distance from the camera the radiance of the direct component depends on the light source spectral power distribution i(λ ), target surface spectral reflectance r(λ ), and the attenuation of light introduced by the medium c(λ ). The relationship is governed by the Beer-Lambert attenuation law [7] ρd(λ ) = r(λ )i(λ )e−dc(λ , (3) where d is the distance light travels through the medium. Total attenuation coefficient The total attenuation coefficient c(λ ) describes how much light at wavelength λ is attenuated as it travels through the medium. Light attenuation depends on how much the medium absorbs light as well as how much light is scattered. The contributions of these two phenomena, denoted a(λ ) and b(λ ) for absorption and scattering respectively, define the total absorption coefficient c(λ ) c(λ ) = a(λ )+b(λ ). (4) Intuitively, the intensity of a particular ray of light traveling through a medium can be decreased either because photons are absorbed by the medium, or because some of the light starts to propagate in different direction as it is reflected off small particles suspended in that medium. Along the ray however the net effect of these two distinct phenomena is the same; light intensity is reduced (Fig. 2). Absorption coefficient In underwater environments, the absorption coefficient is impacted by the optical properties of pure sea water aw(λ ) and the absorption properties of three seawater constituent particles: phytoplankton aΦ(λ ), colored dissolved organic matter (CDOM), aCDOM(λ ), and non-algal particles (NAP), aNAP(λ ). The total absorption coefficient is given by the sum of absorption properties of the constituents a(λ ) = aw(λ )+aΦ(λ )+aCDOM(λ )+aNAP(λ ). (5) 400 450 500 550 600 650 700 0 0.1 0.2 0.3 0.4 0.5 Wavelength, nm a (λ ), a .u . Water